A Comparative Study of Real Workload Tracesand Synthetic Workload

Two basic approaches are taken when modeling workloads in simulation-based performance evaluation of parallel job scheduling algorithms: (1) a carefully reconstructed trace from a real supercomputer can provide a very realistic job stream, or (2) a exible synthetic model that attempts to capture the behavior of observed workloads can be devised. Both approaches require that accurate statistical observations be made and that the researcher be aware of the applicability of a given trace for his or her experimental goals. In this paper, we compare a number of real workload traces and synthetic workload models currently used to evaluate job scheduling and allocation strategies. Our results indicate that the choice of workload model alone { real workload trace versus synthetic workload models { did not signii-cantly aaect the relative performance of the algorithms in this study (two scheduling algorithms and three static processor allocation algorithms). Almost all traces and models gave the same ranking of algorithms from best to worst. However, two speciic workload characteristics were found to signiicantly aaect algorithm performance: (a) proportion of power-of-two job sizes and (b) degree of correlation between job size and job runtime. When used in the experimental evaluation of resource management algorithms, workloads diiering in these two characteristics may lead to discrepant conclusions.